Motion capture systems are a widespread tool in research to record ground-truth poses of objects. Commercial systems use reflective markers attached to the object and then triangulate pose of the object from multiple camera views. Consequently, the object must be visible to multiple cameras which makes such multi-view motion capture systems unsuited for deployments in narrow, confined spaces (e.g. ballast tanks of ships). In this technical report we describe a monocular event-camera motion capture system which overcomes this limitation and is ideally suited for narrow spaces. Instead of passive markers it relies on active, blinking LED markers such that each marker can be uniquely identified from the blinking frequency. The markers are placed at known locations on the tracking object. We then solve the PnP (perspective-n-points) problem to obtain the position and orientation of the object. The developed system has millimeter accuracy, millisecond latency and we demonstrate that its state estimate can be used to fly a small, agile quadrotor.

Over the past decade, camera technology has made gradual, and significant improvements thanks to the mobile phone industry. This has accelerated multiple industries, including Robotics. Today, Davide Scaramuzza discusses a step-change in camera innovation that has the potential to dramatically accelerate vision-based robotics applications. Davide Scaramuzza deep dives on Event Cameras, which operate fundamentally different from traditional cameras. Instead of sampling every pixel on an imaging sensor at a fixed frequency, the "pixels" on an event camera all operate independently, and each responds to changes in illumination. This technology unlocks a multitude of benefits, including extremely highspeed imaging, removal of the concept of "framerate", removal of data corruption due to having the sun in the sensor, reduced data throughput, and low power consumption. Davide Scaramuzza is a Professor of Robotics and Perception at both departments of Informatics (University of Zurich) and Neuroinformatics (joint between the University of Zurich and ETH Zurich), where he directs the Robotics and Perception Group. His research lies at the intersection of robotics, computer vision, and machine learning, using standard cameras and event cameras, and aims to enable autonomous, agile, navigation of micro drones in search-and-rescue applications. Davide Scaramuzza: Hi, thank you. Abate De Mey: So firstly, I'd like to give a little bit of background about why I reached out and invited you to the show today. So over the past few months, I've been working a lot with my team at fluid dev, where we've been building a platform, helping robotics companies scale. And while we were working with one of the companies on that platform, we were digging into a lot of open source VSLAM algorithms. Um, and we just kept running into your name as we were doing research and reading up on this. So I'm super excited to have you on today and I'd love to learn just a little bit more about yourself and what your team is doing.

The auto industry is currently experiencing a rapid shift to autonomous vehicles (AV). This evolution is spearheaded by new, innovative technology companies that are bringing cutting-edge automotive platforms to the market at an unprecedented pace. Currently, vehicles on the road are equipped with the ability to maneuver on their own on highways while in the presence of a human driver. The next logical step in the race to autonomy is self-driving capability in an urban setting -- first with a driver and eventually with humans acting solely as passengers. However, driving in cities is an exponentially more difficult problem to solve than maneuvering on highways.

We demonstrate the first reinforcement-learning application for robots equipped with an event camera. Because of the considerably lower latency of the event camera, it is possible to achieve much faster control of robots compared with the existing vision-based reinforcement-learning applications using standard cameras. To handle a stream of events for reinforcement learning, we introduced an image-like feature and demonstrated the feasibility of training an agent in a simulator for two tasks: fast collision avoidance and obstacle tracking. Finally, we set up a robot with an event camera in the real world and then transferred the agent trained in the simulator, resulting in successful fast avoidance of randomly thrown objects. Incorporating event camera into reinforcement learning opens new possibilities for various robotics applications that require swift control, such as autonomous vehicles and drones, through end-to-end learning approaches.

Self-driving cars promise to make driving a far safer activity, and they'll let occupants use their transportation time for leisure or work tasks. The first company to break through the last remaining barriers to self-driving vehicles stands to benefit tremendously, but who's currently winning? Here are a few of the companies leading the way and some factors that will decide which one wins. Although Google is often credited with advances in self-driving cars, the project is being run by Waymo, which is owned by Google's parent company, Alphabet. Name issues aside, Waymo is widely viewed as the current leader in developing self-driving cars, and for good reason.

Event cameras are revolutionary sensors that work radically differently from standard cameras. Instead of capturing intensity images at a fixed rate, event cameras measure changes of intensity asynchronously, in the form of a stream of events, which encode per-pixel brightness changes. In the last few years, their outstanding properties (asynchronous sensing, no motion blur, high dynamic range) have led to exciting vision applications, with very low-latency and high robustness. However, these sensors are still scarce and expensive to get, slowing down progress. To address this issue, we present ESIM: an efficient event camera simulator implemented in C and available open source.

A few years ago, Davide Scaramuzza's lab at the University of Zurich introduced us to the usefulness of a kind of dynamic vision sensor called an event camera. Event cameras are almost entirely unlike a normal sort of camera, but they're ideal for small and fast moving robots when you care more about not running into things than you do about knowing exactly what those things are. In a paper submitted to Robotics and Automation Letters, Antoni Rosinol Vidal, Henri Rebecq, Timo Horstschaefer, and Professor Scaramuzza present the very first time an event camera has been used to autonomously pilot a drone, and it promises to enable things that drones have never been able to do before. The absolute cheapest way to get a drone to navigate autonomously is with a camera. At this point, cameras cost next to nothing, and if you fuse them with an IMU and don't move very fast and the lighting is reliable, they can provide totally decent state estimation, which is very important.